{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AdaBoost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "boosting是一族将弱学习器提升为强学习器的算法。这族算法的工作机制是：先从初试训练集训练出一个基学习器，再根据基学习器的表现对训练样本进行调整，使得先前基学习器做错的训练样本在后续受到更多关注，然后基于调整后的样本分布来训练下一个基分类器，如此重复进行，直至基学习器数目达到实现指定值T，最后将这T个基学习器进行加权组合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "boosting族算法最著名的代表室AdaBoost,AdaBoost算法有多种推导方式，比较容易理解的是基于“加性模型”，即基学习器的线性组合$$H(x)=\\displaystyle\\sum_{t=1}^T \\alpha_ih_t(x)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "来最小化指数损失函数\n",
    "        $$l_{exp}(H|D)=E_{X D}[e^{-f(x)H(x)}]$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "算法步骤：\n",
    "输入：训练集 $D={(x_1,y_1),(x_2,y_2),..,(x_m,y_m)}$,\n",
    "\n",
    "    基学习算法$\\xi$\n",
    "    \n",
    "    训练轮次T\n",
    "    \n",
    "过程：\n",
    "\n",
    "1.D_1(X)=1/m\n",
    "\n",
    "2.for t =1,2,3...,T do:\n",
    "\n",
    "3.   $h_t=\\xi(D,D_t)$\n",
    "4.   $\\epsilon_t = P_{X~D_t}(h_t(x) ≠ f(x))$\n",
    "5.   if $\\epsilon_t>0.5 then break$\n",
    "6.   $\\alpha_t = \\frac{1}{2}ln(\\frac{1-\\epsilon_t}{\\epsilon_t})$\n",
    "7.   $D_{t+1}(x)=\\frac{D_t(x)exp(-\\alpha f(x)h_t(x))}{Z_t}$\n",
    "8.end for\n",
    "\n",
    "输出：$H(x) = sign(\\displaystyle\\sum_{t=1}^T\\alpha_ih_t(x))$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "其中$\\epsilon_{i}$是每个弱分类器的错误率。$Z_t$是规范化因子，以确保$D_{t+1}$是一个分布\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
